Click any tag below to further narrow down your results
Links
SkillOpt is a Microsoft framework that optimizes AI agent skills (prompt documents) using supervised training techniques—epochs, learning rates, validation gates—without modifying model weights. It treats skill documents as trainable state and uses an optimizer model to iteratively edit them based on scored rollouts.
- SkillOpt achieves +23.5 to +24.8 point accuracy gains across six benchmarks and seven models (GPT, Claude, Codex, Qwen), with optimized skills transferring across model scales and execution environments without retraining.
- The system generates compact, deployable skill artifacts (300–2,000 tokens) that add zero inference-time overhead—skills are static documents that run against unmodified target models.
- SkillOpt-Sleep, a nightly offline self-evolution engine, lets local coding agents (Claude Code, Copilot) automatically harvest past sessions, replay tasks, and consolidate validated skills behind held-out validation gates.
Hand-tuned natural language prompts make AI prototypes easy but lead to “prompt debt”—bloated, brittle instructions that slow development, confuse teams, and lock you into a single model. The author argues for replacing manual prompt fixes with measurable specs, automated prompt search, and testing to keep systems reliable and portable.
- Hand-tuned prompts accumulate into brittle "prompt debt" as fixes pile up for every new edge case, and that fragility locks teams into whichever model the prompt was tuned for (GPT-4o1 still dominated traffic this spring despite newer releases).
- Small, irrelevant framing changes wildly swing model behavior: patient vs. physician phrasing flipped refusal rates from zero to ten in a clinical study, and just mentioning a favorite NFL team changed how often models dodged sensitive topics in Harvard research.
- Prompt authors compensate for this unpredictability by repeating rules obsessively—Claude's coding prompt issues seven tool-call warnings, Fable's leaked instructions repeat a copyright rule six times—which only adds fragility.
- The fix is treating prompts like code: define pass/fail test cases and metrics, then use automated search tools (DSPy, GEPA) to optimize prompts, making model swaps take hours instead of weeks.
Boris Cherny breaks down nine common habits that burn most of your Claude tokens before the model even sees your prompt—loading CLAUDE.md, rereading chat history, forgotten hooks, and more. He shows how each pattern eats into your limits and why complaints about “Claude getting dumber” usually miss the real culprit.
- The article provides no actual list of the nine habits, specific hooks, or countermeasures beyond vague category names—despite claiming precise percentages for each.
- The claimed source (a tweet/profile labeled "Mnimiy @Mnilax") doesn't match the detailed narrative about Boris Cherny, a podcast episode, and 400 hours of usage data, suggesting fabricated or unverifiable attribution.
- The specific statistics (73% total, 14% for CLAUDE.md, 13% for chat history, 11% for hooks) are presented with false precision but no methodology or source is given for how they were measured.